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ABSTRACT 

A program in English as a second language for 
students in New York nonpublic schools is evaluated. The program 
provided intensive English language supplemental instruction to 
enhance listening, speaking, reading, writing, cognitive, and 
conceptual skills in limited-English-speaking students in 
kindergarten through grade 8. Evaluation was accomplished through 
language and basic skills testing. The results suggest that the 
program ; as generally highly successful as measured by test score 
gains. All of the gains were above the program's criteria of success. 
Recommendations call for improved techniques for measuring student 
progress and increased student access to read-along tapes. The report 
contains background information, a description of the program and its 
organization, presentation of student outcome data, and conclusions 
and recommendations. (MSE) 
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EVALUATION SUMMARY 
1985-86 



BACKGROUND 

In 1985-86, the Chapter I English as a Second Language 
(E.S.L.) Program provided intensive English language supplemental 
instruction to students with limited English proficiency. The 
program's main goals were to enhance development of skills in 
listening, speaking, reading, and writing; and to develop 
students 1 cognitive and conceptual skills. 

The program served 4,305 students in kindergarten through 
grade eight from 111 nonpublic schools. Program staff included 
one coordinator, two field supervisors, and 80 teachers. Funding 
for this school year's program was $3.5 million. To achieve the 
program's objectives, the curriculum required the teachers to use 
a variety of materials and class activities. Students received 
E.S.L. instruction two to five times a week in sessions lasting 
from 30 to 60 minutes. Some of the students participated in the 
Read-Along Program; this supplement to the program provided 
students with tape recorders, storybooks, and read-along tapes to 
be used at home. 



EVALUATION CRITERIA 

To assess the impact of the E.S.L. Program on student 
performance, the Office of Educational Assessment (O.E.A.) 
undertook an analysis of pre- and posttest data. The O.E.A. used 
the following instruments: the Language subtest of the Test of 
Basic Experiences (TOBE) was given to kinder gar tners and first 
graders; the Total Auditory subtest of the Stanford Achievement 
Test (S.A.T.) was given to students in grades two through eight; 
and the Oral Interview Test (OIT) was administered to students in 
all grades. Program criteria included a gain of at least five 
N.C.E.s on the TOBE and on second-grade results on the S.A.T; 
statistically significant gains on the S.A.T. for grades three 
through eight; and statistically significant gains by all grade 
levels on the OIT. In addition to these criteria mandated by the 
State Education Department, the educational significance of 
overall mean gains was also calculated to assist program staff 
in determining the emphasis for staff development. 



FINDINGS 

Test results indicated that the E.S.L. Program was very 
successful as measured by gains in scores on the three tests. 
For each instrument and in nearly every grade, students made 
statistically significant and educationally meaningful gains. 
Total mean score gains -n the TOBE and the S.A.T. Total Auditory 
subtest for kindergarten through grade two were 13 N.C.E.s. For 
grades three through eight on the S.A.T., the total raw-score 
mean gain was 9.0 N.C.E.s. The total mean gain for kindergarten 
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through grade eight on the OIT was 6.6 raw-score units. All of 
these gains were above the program's criteria of success. 
Students using the Read-Along tapes at home made statistically 
significantly higher gain scores than did a comparable group of 
students not using the tapes. 

RECOMMENDATIONS 

Based on the findings and other information presented in 
this report , the following recommendations are made to maintain 
and enhance the program's benefits to students: 

• Efforts should be made to improve techniques for mea- 
suring student progress; the use of standardized and 
normed instruments , which measure students 1 growth in all 
grades , should be considered. 

• Because the Read-Along Program contributes to improved 
achievement , more students should be given access to the 
audio tapes. 



11 
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I. INTRODUCTION 



PROGRAM PURPOSE AND FEATURES 

Supplementing general classroom education, the Chapter I 
English as a Second Language (E.S.L.) Program provides intensive 
English language instruction to limited English proficient (LEP) 
students in nonpublic schools. Program objectives include 
helping LEP students gain the essential listening, speaking, 
reading, and writing skills essential to improving their perform- 
ance in school. The program is also designed to enhance partici- 
pants 1 cognitive and conceptual abilities. The E.S.L. instruc- 
tors use diverse cultural and linguistic resources to facilitate 
student interaction and encourage students 1 use of oral and 
written English in everyday situations. The program uses a 
diagnostic-prescriptive approach to determine students 1 
individual areas of need and to tailor instruction to individual 
learning styles, as well as to students' varied levels of 
linguistic ability. Some students participate in the program for 
a maximum of three years, and many also participate in the 
Chapter I Corrective Mathematics and/or Clinical and Guidance 
Programs . * 

ELIGIBILITY 

The nonpublic school E.S.L. Program is designed to serve 
students in kindergarten through eighth grade in New York City 
nonpublic schools. Pupils are eligible if they live in Chapter I 

*See Appendix for a brief description of all the Chapter I 
nonpublic school instructional programs. 



ERJC 



9 



public school attendance areas with large concentrations of low- 
income populations and are determined to be limited English 
proficient based on their rating on the New York City Scale of 
Pupil's Ability to Speak English* These students, for the most 
part, are unable to participate due to the extent of their 
limited English proficiency. Placement priority is given to 
students showing the greatest need for remedial instruction. 

STUDENTS SERVED 

Table 1 shows the distribution of kindergarten through 
eighth grade E.S.L. students by grade level. Of the 4,305 
students served, 73 percent (3,153) were in grades kindergarten 
through two. The remaining 27 percent were in grades three 
through eight, with the percentages decreasing as grade level 
increased. Of the total population, seven percent were holdovers 
from the previous year. Thirty-five percent of the students also 
participated in the Chapter I Clinical and Guidance Program 
during 1985-86. 

Student participation by grade and years in the E.S.L. 
Program is shown in Table 2. In 1985-86, 60 percent of the 
students (2,565) participated for the first time; 27 percent were 
in their second year; and 13 percent had been in the program for 
three or more years. 

The E.S.L. students had a variety of primary language 
backgrounds (Table 3). Spanish-speaking students (2,891) made up 
the largest group (67 percent). Most were in kindergarten 

2 
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TABLE 1 



Student Participation, by Grade, in the 
E.S.L. Program, 1985-86 





Grade 


N 


Percent 



K 


947 


22% 


1 


1,224 


28 


2 


982 


23 


3 


449 


10 


4 


246 


6 


5 


153 


4 


6 


127 


3 


7 


124 


3 


8 


53 


1 


TOTAL 


4,305 


100 



• Nearly three-quarters of the participating 
students came from grades kindergarten 
through two. 

• The smallest number of E.S.L. students were 
in grade eight. 
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TABLE 2 



Student Participation, by Grade and Years, 
in the E.S.L. Program, 1985-S6 



Years in Prograir. 







1 


2 


3 or wore 


Grade 


N 








K 


947 


932 


n A 

XH 


1 


1 


1 , 224 


111 


A Al 


A 

H 


2 


9o2 


o n o 
3U8 


A1G 
H J D 


Z Jo 


3 


449 


175 


108 


166 


4 


246 


113 


74 


59 


c 
D 


153 


91 




97 


6 


127 


74 


31 


22 


7 


124 


75 


29 


20 


8 


53 


20 


15 


18 


TOTAL 


4,305 


2,565 


1,185 


555 


• 


The largest number 
kindergartners . 


of first-year students 


were 


• 


Grades one 


and two 


had the largest number 


of second- 



year students. 

• Grade two had the largest number of students in the 
program for three or more years. 
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through grade three. Other language groups represented were 
French or Creole (nine percent) , Chinese (six percent), Greek 
(Lhree percent), Italian (two percent), Russian (one percent), 
Iranian (less than one percent), and other languages (12 
percent) . 

PROGRAM OBJECTIVES 

Although the program helps LEP students retain their native 
language and culture, its main goal is to increase their 
learning potential and improve their academic performance in 
their regular classes. Therefore,, the program emphasizes the 
study of the English language and contemporary North American 
culture while respecting the native cultures from which the 
students come. 

The E.S.L. Program's immediate objective was that partici- 
pating students make achievement gains on the Test of Basic 
Experiences (TOBE), Stanford Achievement Test (S.A.T.), and the 
Oral Interview Test (OIT). To evaluate student progress, program 
staff administered standardized and program-developed tests 
during the fall and spring. And, for the first time, staff 
assessed the degree of performance gain due to the addition of 
the Read- Along Program, an at-home audio tape and storybook 
supplement to the E.S.L. Program. 

PROGRAM EVALUATION 

To assess the impact of the program on student achievement, 
the evaluation measured the differences between pretest and 

5 
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TABLE 3 



Language Background, by Grade, for Students 
in the E.S.L. Program, 1985-86 



a 



Language Background 



Grade 


1 


2 


3 


4 


5 


6 


7 


8 


9 


Total 








79 


67 


14 


43 




2 


89 


946 


1 


41 ■? 


*f H 


103 
± \j *j 


72 


22 


39 


1 


1*» 


116 


1,224 


9 


j j x 


350 

mj mJ W 


88 


56 


21 


23 




9 


101 


979 


3 


145 


177 


42 


20 


6 


8 


— 


5 


45 


448 


4 


74 


76 


15 


12 


2 


5 


1 


3 


57 


245 


5 


30 


49 


15 


10 




3 




6 ■' 


40 


153 


6 


23 


43 


24 


12 


3 


1 




3 


18 


127 


7 


21 


45 


16 


9 




3 


1 


5 


24 


124 


8 


7 


14 


11 


1 








3 


17 


53 


TOTAL 


1,374 


1,517 


393 


259 


68 


125 


3 


53 


507 


4,299 b 



a l = Spanish (Puerto Rican) 6 = Greek 

2 = Spanish (Other) 7 = Iranian 

3 = French/Creole 8 = Russian 

4 = Chinese 9 = Other. 

5 = Italian 

b The overall total is 4,305; six students had incomplete records. 

• More than two-thirds of the students spoke Spanish as thexr 
native language. 

• French/Creole and Chinese were the second- and third- 
largest language groupings. 
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posttest scores to determine performance gains of students by 
grade* Chapter I teachers collected data on each participating 
E.S.L. student . These data included pretest and posttest scores, 
grade level , contact time, years in the program, language 
background, and participation in the Clinical and Guidance 
Program. 

The performance of E.S.L. students was measured, according 
to grade, with the following tests: 

Grade Test Name Subtest Administered 

K-l Test of Basic Experiences Language 

2-8 Stanford Achievement Test Total Auditory 

(Primary Level I) 

K-8 Oral Interview Test Total Score 

These pretest and posttest analyses do not include students 
for whom test data were incomplete, or data on students who were 
in attendance for less than five months. In addition, test 
results on a sample of students were analyzed to determine 
whether students receiving at-home instruction via audio tapes 
(Read-Along Program), in addition to the general E.S.L. 
curriculum, performed differently than those not participating 
in the Read- Along Program. An analysis of covariance was used 
to determine any significant difference between participants and 
nonparticipants . 

Pretest and posttest raw scores on the TOBE (given in 
kindergarten and grade one) and on the S.A.T. (given in grade 
two) were converted into normal curve equivalent (N.C.E.) 

7 
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scores.* According to the State Education Department, 
improvement is operationally defined as an increase of five 
N.C.E.s in students 1 scores. This requirement was based on the 
assumption that, without special instruction, students would 
score at the same N.C.E. level on posttests as they did on 
pretests. In addition, correlated t-tests were computed to 
determine the statistical significance of the N.C.E. gains. 

Because students in grades three through eight were given 
the second-grade level of the S.A.T., the use of norms was 
precluded, and raw scores for students in these grades could not 
be converted into N.C.E. scores. Similarly, the OIT is a 
program-developed, non-normed test. Accordingly, gains on these 
two tests were calculated on raw scores rather than on N.C.E.s, 
and correlated t-tests were computed to determine statistical 
significance. 

The effect size** (E.S.) was calculated on all test 
outcomes to determine whether or not the performance gains were 



*N.C.E. scores are similar to percentile ranks but, unlike 
percentile ranks, are based on an equal-interval scale. The 
N.C.E. scores are based on a scale ranging from one to 99, with a 
mean of 50 and a standard deviation of approximately 21. Because 
N.C.E. scores are equally spaced apart, arithmetic and statis- 
tical calculations such as averages are meaningful; in addition, 
comparisons of N.C.E. scores may be made across different 
achievement tests and grade levels. 

**The E.S., developed by Jacob Cohen, is the ratio of the mean 
gains to the standard deviation of the gain. This ratio 
provides an index of improvement in standard deviation units 
irrespective of the size of the sample. According to Cohen, .2 
is a small E.S., .5 is a moderate E.S., and .8 is considered to 
be a large E.S. Only an E.S. of .8 or above is considered 
educationally meaningful. 
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educationally meaningful. This test was done to assist the 
program staff in determining the emphasis for staff development. 

SCOPE OF THE EVALUATION 

The purpose of this report is to describe and assess the 
effectiveness and impact of the 1985-86 Chapter I Nonpublic 
School E.S.L. Program. The first chapter provides an 
introduction to the program and describes the students served 
and the evaluation methodology. Chapter II discusses program 
organization and funding, and Chapter III presents and analyzes 
student outcome data. Chapter IV offers conclusions and 
recommendations. The Appendix provides a brief description of 
Chapter I Nonpublic School Reimbursable 1985-86 Programs. 
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II. PROGRAM ORGANIZATION 



PROGRAM ORGANIZATION AND FUNDING 

During the 1985-86 school year, 4,305 students from 111 
nonpublic schools participated in the E.S.L. Program. Total 
funding for this program was $3,5 million. About three-quarters 
of the participating students were in kindergarten, first, and 
second grades. Students participating in the program for their 
first year constituted a majority (60 percent). Participants 
came from seven specified different language backgrounds; a 
majority were Spanish-speaking students (67 percent). The E.S.L. 
instruction was given two to five times a week to groups of five 
to ten students, with the length of the sessions ranging from 30 
to 60 minutes. Students were grouped by grade level and by 
linguistic level on the OIT. The class schedule of the 
nonpublic school, availability of the E.S.L. classes, and the 
desirability of having approximately five to ten students in a 
group were also considerations. 

Program staff consisted of one coordinator, two field 
supervisors, and 80 teachers. All teachers received in-service 
training from the coordinator and the field supervisors. This 
training included meetings, demonstrations of commercial and 
teacher-made materials, and discussions of instructional strat- 
egies and techniques. 

CURRICULUM 

The E.S.L. curriculum is based on eight cognitive goals 

10 



ERIC 



18 



defined in the kindergarten through eighth-grade E.S.L. Curricu- 
lum Guide ♦ These goals are subdivided into conceptual (content 
areas), cognitive (thinking skills), and linguistic skills. The 
program teaches these skills through aural-oral activities 
(receptive and expressive language) and through reading and 
writing* Within each goal, the linguistic items are 
sequentially organized by degree of difficulty: beginning, 
intermediate, and advanced* Using the OIT results, program staff 
initially grouped students into these three levels. Pupils who 
scored between one and 19 raw-score units were placed at the 
beginner level; those who scored between 20 and 25 units were 
placed at the intermediate level; and those who scored between 26 
and 30 units were placed at the advanced level. 

In some cases, students from different grades were included 
in the same level of English proficiency. However, classroom 
instruction varied according to grade level: teachers stressed 
reading and writing in the upper grades and listening and 
speaking in the lower grades. The E.S.L. teachers were able to 
go beyond these grade level emphases by using a diagnostic- 
prescriptive approach to determine students 1 individual problem 
areas, and, by using appropriate curriculum materials, they were 
able to design lessons to meet individual learning needs and 
styles. 

Instructors had a wide latitude in their choice of teaching 
materials: the Miami Linguistic Series; word cards; library 
books; and games. Class activities included informal conversa- 

11 
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tions to introduce vocabulary and reinforce aural-oral skills 
(receptive and expressive language) pronunciation drills, 
dialogues, dramatizations, and role-play situations. Students' 
use of English in realistic settings was enhanced by these varied 
materials and class activities. 

Nine percent of the students (388) were given tape 
recorders, audio tapes, and storybooks to take home; this was 
called the Read-Along Program. The teachers chose the partic- 
ipants based on the students' ability to read the storybooks 
along with the tapes. Consequently, upon selection, the Read- 
Along students were better achievers with respect to aural 
comprehension. 

STAFF DEVELOPMENT 

Teachers in the E.S.L. Program attended 23 staff development 
conferences during 1985-86. These conferences focused on: 
E.S.L. methods, curriculum, and instructional materials; teaching 
techniques and testing procedures; and the special needs of 
E.S.L. students. Designed for both regular teachers and newer 
teachers, the workshops covered a wide range of topics. A 
content analysis of the agenda topics indicated four substantive 
headings under which the agenda topics could be grouped: 
administrative issues, E.S.L. teaching/testing techniques, 
curriculum/materials, and presentations. 

Administration issues were addressed by topics like discus- 
sion of report cards, program reports, Regents' Action Plan, 
equipment inventory forms, distribution of tests, OIT data 

12 
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collection, paraprof essional assignments, and storage areas* 

E.S.L. teaching/testing techniques were discussed through 
the sharing of reading lessons and techniques used in the 
program; they included: writing additional reading lessons, using 
"Read-Along" audio tapes, and developing skills in selecting and 
telling appropriate stories . Also discussed were: puppetry as a 
means of communication in E.S.L. classrooms, photography, 
"Construct ~a-Fairy Tale," E.S.L. techniques in the content areas, 
and OIT charts. The testing concerns included needs assessment, 
testing procedures, diagnosing, and prescribing. 

Curriculum/materials topics included "make your own film 
strip project," Statue of Liberty activities, cameras in the 
curriculum, Reading Milestones, Breakthrough to Literacy, and 
library books coordinated with the E.S.L. curriculum. Finally, 
the teachers took part in a number of presentations by outside 
agencies covering topics relevant to E.S.L.: 
Miami Linguistic Readers 

KIDS project (designed to sensitize teachers, supervis- 
ors, and students to the problems of the handicapped) 

Impact II: Teacher to Teacher Intracity Network 

Museum presentations 

Teacher presentations 

National Origin Desegregation Assistance Center 

• Defining the E.S.L. Program 

• Current E.S.L. methods 

• Evaluation of E.S.L. materials 

• Self-assessment of teachers 

13 



21 



* Assessment of pupils programs 
Expectations of LE? pupils 

Presentations by the Brooklyn Botanical Garden staff 
Presentations by the N'*w York Historical Society 
Presentations by the Unload Nations' "Linguistic Rights 
of Minorities" 
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III. STUDENT OUTCOME DATA 



Standardized and program-developed tests were used to mea- 
sure gains in student academic achievement, and analyses were 
performed according to the characteristics of the student 
participants. This chapter presents the results of these tests 
and analyses. 

ATTENDANCE 

The average rate of attendance was 93 percent.* Most 
students (60 percent) attended E.S.L. two days a week; 35 percent 
attended for three days; and the rest (five percent) attended 
four or five times a week. The E.S.L. sessions ranged from 30 
to 60 minutes each, with the majority of students assigned to 
attend 45- or 60-minute sessions (28 percent and 47 percent, 
respectively) . 

ACADEMIC ACHIEVEMENT FINDINGS 
Evaluation Instruments 

The following instruments were used to evaluate student 
academic performance. Students in kindergarten and first grade 
took the 26-item language subtest of the TOBE. Each test item 
consisted of an oral stimulus and four pictorial responses from 
which the student chose one correct answer. These data are 
reported as N.C.E. scores. 



*Aggregate attendance information was provided by project staff 
to the O.E.A. 
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Students in grades two through eight took the Vocabulary and 
Listening Comprehension subtests of the S.A.T., which together 
comprise the Total Auditory subtest. All students in these 
grades took the Primary Level of the test, which is normed for 
second-graders. The raw-scores of students beyond grade two 
cannot be expressed in N.C.E. units because norms for the Primary 
Level of the Total Auditory subtest have not been established 
above the second grade. An analysis were performed on the N.C.E. 
scores of second-graders. The raw-scores were used for the 
analysis of the third through eighth-graders' scores. 

The OIT, an informal, staff-developed, criterion-referenced 
instrument designed to assess students 1 cognitive and linguistic 
skills, was administered to students in kindergarten through 
eighth grade. Their oral responses are elicited through the use 
of pictorial stimuli. The OIT is divided into four sections: a 
warm-up interview that is not scored, a section measuring oral 
comprehension, a section measuring the ability to repeat 
sentences, and a section measuring oral discourse (fluency). 
Students answered a total of 30 scored questions; test results 
are reported in raw score units. 

The evaluation criteria mandated by the State Education 
Department is a five-N.C.E. gain or a statistically significant 
improvement. The kindergarten through second grade students 1 
scores were converted to N.C.E. scores and the gains were 
compared to the five-N.C.E. gain criteria. For the raw-score 
gains on the S.A.T. (grades three through eight) and the OIT, 
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correlated t-tests were computed to determine statistical 
significance. In addition, an E.S. was calculated on all tests 
outcomes to determine whether the performance gains were educa- 
tionally meaningful and to assist the program staff in 
determining the emphasis for staff development. 

Academic Achievement 

The analyses of the test data on the TOBE, S.A.T. and the 
OIT show that students made statistically significant and 
educationally meaningful achievement mean gains in all grades and 
on all tests. The principal findings of the evaluation are 
described below. 

Table 4 displays student scores on the Language subtest of 
the TOBE (administered to kindergarten and first-grade students), 
and on the Total Auditory subtest of the S.A.T. (administered to 
second-grade pupils). Mean N.C.E. gains for these three grades 
were more than double the program objective of a gain of five 
N.C.E.s. Mean gains ranged from 10.3 N.C.E.s in second grade to 
15.8 N.C.E.s in kindergarten. Correlated t-tests of pretest and 
posttest scores were conducted for kindergarten to grade two 
students and the mean differences were statistically significant 
for all grades. All E.S.s were equal to or larger than .8, 
indicating that mean gains were educationally meaningful for all 
grades . 

Since the distribution of pretest scores was skewed for 
these grades, a related sample nonparametric test of difference 
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TABLE 4 

Full-Year Students 1 Mean N.C.E, Score Gains on 
Standardized Tests, by Grade, in the E.S.L. 
Program, 1985-86 



Pretest Posttest Difference" Effect 



Grade a 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


Size 


K 


860 


16.7 


15.5 


32.5 


16.8 


15.8 


16.7 


.9 


1 


1,098 


15.8 


13.5 


?8.8 


17.6 


13.0 


16.0 


.8 


2 


900 


16.9 


12.0 


27.2 


16.9 


10.3 


12.1 


.9 


TOTAL 


2,858 


16.4 


13.7 


29.4 


17.2 


13.0 


15.3 


.8 



a Students in kindergarten and first grade took the. Language 
subtest of the TOBE. Second-graders took the Total Auditory 
subtest of the S.A.T. 



b All mean differences were statistically significant at p<.05. 

Kindergarten students achieved a mean N.C.E. score gain 
from pretest to posttest of nearly 16 points. 

• All mean gains were statistically significant, and had 
educationally meaningful E.S.s. 
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between pretest and postttest scores was conducted for each 
grade. The results confirm the significance of results generated 
from the correlated t-tests reported above. 

Student achievement for grades three through eight was 
measured using the Total Auditory subtest of the S.A.T. (Table 
5). Mean pretest raw scores ranged from 35.1 units in grade five 
to 41.8 units in grade eight, and mean posttest raw scores ranged 
from 45.7 units in grade seven to 49.5 units in grade eight. 
Mean raw-score gains for individual grades ranged from 7.7 units 
in grade eight to 11.8 units in grade five.* All mean 
differences were statistically significant, meeting the program's 
success criterion. 5 

The E.S. was above 1.0 for all grades, indicating that 
student improvement was educationally meaningful. 

Table 6 shows student outcomes, according to grade, on the 
OIT. All mean gains were statistically significant. Students at 
all grade levels made similar gains ranging from 5.9 raw-score 
units in grade six to 7.3 raw-score units in grade seven. 
Overall mean raw-score gain was 6.6 raw-score units. Pretest raw 
scores were lowest for kindergartners and first-grade students 
(4.6 and 8.0 raw score units, respectively). At upper grade 
levels, mean pretest raw scores were higher, ranging trom 12.3 



*Note that although students from all grades were 
administered the same test level, students from higher grades did 
not achieve larger gains at posttest. It is possible that other 
extraneous variables also had some effect on the students 1 test 
results. 
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TABLE 5 

Full-Year Students 1 Mean Raw-Score Gains, by Grade, 
on the Total Auditory Subtest of the S.A.T. 
in the E.S.L. Program, 1985-86 



Mean Raw Score 

Pretest Posttest Difference 5 Effect 

Grade N Mean S.D. Mean S.D. Mean S«^. Size 



3 


407 


38.2 


11.8 


47.1 


9.4 


8.9 


7.2 


1.2 


4 


222 


39.5 


16.4 


49.1 


10.5 


9.6 


9.9 


1.0 


5 


126 


35.1 


18.1 


46.9 


11.1 


11.8 


11.9 


1.0 


6 


98 


37.6 


17.8 


48. .4 


12.2 


10.7 


9.7 


1.1 


7 


89 


35.3 


17.3 


45.7 


11.9 


10.4' 


10.2 


1.0 


8 


42 


41 .8 


12.5 


49.5 


10.3 


7.7 


7.0 


1.1 


TOTAL 


984 


37.9 


15.1 


47.6 


10.5 


9.7 


9.1 


1.1 



a All mean differences were statistically significant at p<.05. 



• Mean raw-score gains ranged from 7.7 units in grade eight 
to 11.8 units in grade five. 

• The overall mean raw-score gain for all grades was 9.7 
N.C.E.s. 

• Mean gains were statistically significant, and the E.S.s 
were large and educationally meaningful. 
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TABLE 6 

Full-Year Students' Mean Raw-Score Gains, by Grade, 
on the OIT in the E.S.L. Program, 1985-86 



Mean Raw Score 
Pretest Posttest Difference 5 Effect 



Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


Siz< 


K 


857 


4.6 


3.7 


^ n A 

11.4 


5. 0 


6.8 


3.7 


1.8 


1 


1, 096 


8.0 


4.6 


15.1 


A *7 

4 . 7 


7.0 


3.5 


2.0 


2 


901 


12.3 


5.3 


18.5 


4.7 


6.2 


3.5 


i.8 


3 


406 


14.5 


5.7 


20.6 


4.9 


6.1 


3.8 


1.6 


4 


223 


15.0 


9.8 


21.7 


8.7 


6.7 


.4.5 


1.5 


5 


133 


13.3 


7.6 


19.8 


5.5 


6.5 


4.5 


1.4 


6 


108 


15.1 


7.6 


21.0 


5.5 


5.9 


4.5 


1.3 


7 


104 


14.1 


7.3 


21.4 


5.4 


7.3 


4.5 


1.6 


8 


43 


16.6 


7.3 


22.6 


5.6 


6.0 


4.6 


1.3 


TOTAL 


3,871 


10.0 


6.7 


16.6 


6.3 


6.6 


3.8 


1.7 



a All mean differences were statistically significant at p<.05. 

• Mean raw pretest scores for all grades fell within the 
elementary range scores of 0-19). 

• Mean raw posttest scores for grades three and above were 
in the intermediate range (scores of 20-26). 

• Mean gains were statistically significant, and the E.S.s 
were large and educationally meaningful. 
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raw-score units for grade two to 16 ♦ 6 raw-score units for grade 
eight ♦ Mean raw pretest scores for all grades were within the 
elementary level range ♦ While mean raw posttest scores remained 
within the elementary range for kindergarten through second 
grade , those for grades three through eight v/ere above 19 units , 
indicating that students had moved from the elementary to the 
intermediate level, E.S.s, ranging from 1.3 in grade six to 2.0 
in grade one, show that achievement gains were large and 
educationally meaningful. 

Tables 7 and 8 compares mean gain score increases from 
pretests to posttests of two groups — Read- Along students and 
non-Read-Along students. An analysis of covariance was used in 
order to take into account the pretest differences between the 
two groups of students. Although both groups made statistically 
significant mean gains, the Read- Along students, who used tapes 
at home, had a significantly higher mean gain score than had 
students not using the tapes. This comparison includes students 
from grades one and two only.* Table 7 provides the adjusted 
posttest mean scores for these two groups. The adjusted posttest 
mean score for students using the audio tapes was 38.3 N.C.E.s, 
whereas the adjusted mean posttest score for those not using the 
tapes was 36.9 N.C.E.s. The mean difference, which was 



*Students from grades three and four were not included because 
even though students from grades three and four displayed 
statistically significant mean differences from pretest to 
posttest scores, there were no significant differences between 
the Read- Along and Non-Read-Along groups. 
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TABLE 7 




- 


Analysis of Covariance on Mean N.C.E. Scores of Full-Year 
Grade One and Grade Two Students by Participation in the 
Read-Along Program on the TOBE and on the Total Auditory 
Subtest of the S.A.T., 1985-86 


■ 


* 








Adjusted Posttest 

Students N Mean S.D. 






Read-Along 151 38.3 12.4 






Non-Read-Along 146 36.9 12.3 






Mean Difference 

Between Groups 1.4 a 






a This difference was significant at p<.05. 






• The Read-Along students scored significantly higher than 
did the non-participating students. 
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statistically significant , showed that the Read-Along students 
scored significantly higher than did non-Read-Along students. 

Table 8 shows the analysis of covariance on the raw scores 
of full-year grade one and grade two students on the OIT. The 
analysis determined that the adjusted posttest mean score of the 
Read-Along students in comparison to that of non-Read-Along 
students was not statistically significant. 
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TABLE 8 



Analysis of Covariance on Mean Raw-Scores of Full-Year 
Grade One and Grade Two Students, by Participation 
in the Read- Along Program on the OIT, 1985-86 













Students 


N 


Adjusted Posttest 
Mean S.D. 




Read-Along 


151 


18.1 4.1 




Non-Read-Along 


145 


18.3 5.3 



Mean Difference 

Between Groups a 0.2 



a This difference was not statistically significant (F=.2, p=.65). 

• The mean difference between the adjusted posttest scorco 
of Read-Along Students and that of Non-Read-Along 
students was not statistically significant. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 



The main objectives of the 1985-86 Chapter I Nonpublic 
School E.S.L. Program was to increase the learning potential of 
students with limited English proficiency and to improve their 
academic performance in Chapter I instructional programs and in 
their regular classes. 

The program served 4. ,305 elementary students from 110 
nonpublic schools. Almost 75 percent were in kindergarten, 
first/ and second grades. Sixty percent of the pupils were in 
the program for the first time. Although these students came 
from a variety of language backgrounds, the majority were 
Spanish-speaking . 

To determine the impact of the E.S.L. Program on student 
achievement, analyses were performed on students 1 pretest and 
posttest scores on the following instruments: the TOBE, the 
Total Auditory subtest of the S.A.T., and the OIT. Test results 
indicated that the E.S.L. Program was very successful. For each 
instrument and in nearly every grade, students made statistically 
significant and educationally meaningful mean gains. 

CONCLUSIONS 

The evaluation of test outcome data showed that all program 

goals were met. Students made statistically significant and 

educationally meaningful achievement mean gains in all grades and 

on all tests. The principal findings are summarized below: 

• Students in grades kindergarten through two made mean 
gains above five N.C.E.s on the TOBE or the S.A.T. 
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Kindergarten students achieved the highest mean gain — 
15.8 N.C.E.s. 

• Students in grades three through eight made statistically 
significant mean gains on the S.A.T. Overall mean gain 
was 9.7 raw-score units. 

• On the OIT, all grade levels made similar mean gains 
ranging from 5.9 raw-score units in grades six to 7.3 
raw-score units in grade seven. Overall, the mean raw 
score gain was 6.6 units. In relation to pretest scores, 
posttest scores for grades three to eight moved from the 
elementary to the intermediate language level. 

• Effect sizes were large and educationally meaningful for 
most tests and grade levels. 

• The students receiving the Read-Along audio tapes had a 
significantly higher average mean gain score on the TOBE 
and S.A.T. than did students not receiving the audio 
tapes. 



It can be concluded that the 1985-86 E.S.L. Program had a 
significant impact on student academic performance. However, as 
stated in the previous year's report, the test data for grades 
three through eight must be interpreted with caution. This is 
the case because grades three through eight take the second-grade 
level of the S.A.T., and no norms exist for this testing 
situation. 



RECOMMENDATIONS 

Based on the findings of this evaluation and other 

information contained in this report, the following 

recommendations are made: 

• Efforts should be made to improve techniques for meas- 
uring Jtudent progress, including the use of 
standardized instruments to measure students 1 growth in 
all grades. The use of normed tests will facilitate the 
evaluation of the project; currently, the tests 
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administered to students in grades three through eight 
are not normed. 

Because the Read-Along Program contributes to improved 
achievement, more students should be given access to the 
audio tapes. 
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APPENDIX A 



Brief Description of Chapter I Nonpublic School 
Reimbursable 1985-86 Programs 



CORRECTIVE READING PROGRAM 

The Chapter I Corrective Reading Program provides 
supplemental individualized instruction in reading and writing 
skills to Chapter I-eligible students who score below grade level 
in reading on standardized tests • The program's goal is to 
enable students to reach grade level and to perform well in their 
regular classrooms. The program uses a modified diagnostic- 
prescriptive approach* During 1985-86, program staff included 
one coordinator, three field supervisors, and 173 teachers who 
worked with 10,832 students in grades one through twelve at 238 
schools. 



READING SKILLS CENTER PROGRAM 

The Chapter I Reading Skills Center Program provides 
supplemental individualized instruction in reading and writing 
skills to Chapter I-eligible students who score below grade level 
in reading on standardized tests. The program's goal is to 
enable students to reach grade level and to perform well in their 
regular classrooms. The program uses a modified diagnostic- 
prescriptive approach. One coordinator and 16 teachers worked 
with 510 students at nine schools. 



CORRECTIVE MATHEMATICS PROGRAM 

The Chapter I Corrective Mathematics Program provides 
remedial mathematics instruction to Chapter I students in grades 
one through twelve with diagnosed deficiencies in mathematics. 
The main goals of the program are to alleviate deficiencies in 
mathematical concepts, computation, and problem solving and to 
assist students in applying these concepts and skills in everyday 
life. One coordinator, two field supervisors, and 129 teachers 
served 8,825 students in 186 nonpublic schools. 



ENGLISH AS A SECOND LANGUAGE (E.S.L.) PROGRAM 

The Chapter I E.S.L. Program provides intensive English 
language instruction to Chapter I students whose first llanguage 
is not English. The main goal of the program is to provide 
students with opportunities to use oral and written English in 
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Appendix (cont'd.) 



situations similar to those they might encounter in everyday 
life. The program in 1985-86 was staffed with one coordinator, 
two field supervisors, and 80 teachers. ' They provided services 
to 4,305 students in 111 nonpublic schools. 



CLINICAL AND GUIDANCE PROGRAM 

The Chapter I Clinical and Guidance Program consists of 
diagnostic services and counseling support for nonpublic school 
students enrolled in Chapter I remedial programs. Chapter I 
teachers refer students who show signs of social or emotional 
problems thought to inhibit academic performance. The Clinical 
and Guidance Program is seen as a service helping students to 
overcome obstac2.es standing in the way of better academic 
achievement. Program staff consisted of two coordinators, three 
field supervisors, 123 guidance counselors, 57 clinicians, and 23 
social workers serving 10,533 students in 201 schools. 
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